Searching PAJ 



2> 



Seite 1 von 




1/1 



PATENT ABSTRACTS OF JAPAN 

(1 1 )Publlcation number : 03-099952 
(43)Date of publication of application : 25.04.1991 



(51)lnt.CI. 




. B60R 1/00 
B60R 21/00 
H04N 7/18 






•v— 

£21) Application number : 
\*2)Date of filing: 


01-234612 
12.09.1989 


(71) Applicant : 

(72) lnventor : 


NISSAN MOTOR CO LTD 
NOSO KAZUNORI 







4 



(54) SURROUNDING SITUATION MONITOR FOR VEHICLE 

(57)Abstract: 

PURPOSE: To correctly recognize the relative position to 
obstacles around a vehicle by converting the images from 
cameras installed on the vehicle into other coordinates by 
perspective conversion, synthesizing the converted images 
into one image in relation with the image of the vehicle, and 
display it on a display. 

CONSTITUTION: Multiple cameras 1-6 are buried on a 
vehicle 10 to cover the surrounding environment of the 
©licle 10 with respective irradiation ranges (visual fields) 1a- 
6a, eg., two each on front and rear bumpers and one each on 
front winkers. Images from cameras 1-N are inputted to an 
image converting section 7 and converted into other 
coordinates b ypei^efctiv e^verslon then synthesized into 
one image byan image display section 9. The synthesized 

image is displayed on a TV monitor 9 installed at a driver's seat. A high-speed processor capable 
of processing the images of cameras in parallel is stored in the image converting section 7. The 
image display section 8 concurrently depicts the position of the vehicle 10. and the display pos.t,on 
is displaced according to the operation state. 
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PROFESSIONAL SPECIFICATION 

1 . TITLE OF THE INVENTION 
VEHICLE SURROUNDING MONITOR 

2. CLAIMS 

A vehicle surrounding monitor, characterised by having one or multiple camera 
units mounted on a vehicle; means of transforming images captured by said 
camera units into other coordinate system using perspective transformation; 
means of compositing multiple transformation images into a single image relative 
to the image of the vehicle; and a display that shows said image to an occupant 
of the vehicle. 

3. DETAILED DESCRIPTION OF THE INVENTION 
[The Purpose of the Invention] 

(Industrial Application) 

This invention relates to an apparatus for displaying the information of vehicle 
surroundings including the position of other vehicles, obstacles, and the road 
center-line for the driver by using television (TV) camera units (herein called as a 
"camera unit", however any types of camera may be used as long as they 
capture pictures. 
(Description of the Prior Art) 

As conventional vehicle monitors, for example a monitor shown in Fig. 5 is 
available. This is a monitoring system in which the camera unit 21 is installed in 
the vehicle interior facing rearward as shown in Fig. 5 (a) to capture an image, 
which is mirror reversed by the picture signal processor 22 (Fig. 5 (b)), and is 
displayed in the monitor TV 23, obtaining so-called a "rear view image" as shown 
in Fig. 5 (c). Viewing this image enables the driver to determine whether it is safe 
to drivebackward_^ 
(Problem(s) to be solved by the Invention) 

However, with such a conventional vehicle monitor described above, which is a 
device simply displays a rear view image as shown in Fig. 5 (c), it is hard for the 
driver to have a sense of distance between the vehicle and an obstacle, wherein 
a single camera unit is insufficient to gain situational awareness of the vehicle's 
surroundings, and moreover, it is difficult for the driver to sufficiently gain the 
situational awareness of surroundings without being aware of the correlation of 
every camera units utilized when the number of monitors is increased by adding 



more camera units, and consequently, there has been a problem with the driver 
unable to take appropriate driving measures. This invention aims at offering a 
vehicle surrounding monitor which allows the driver to be sufficiently aware of the 
positional relationship between the vehicle and the surroundings whereby the 
driver can take appropriate driving measures, by which the aforementioned 
problem is solved. 
[Constitution of Invention] 
(Means for Solving the Problem) 

In order to attain this purpose, this invention is constituted by using one or 
multiple camera units mounted on a vehicle; means of transforming images 
captured by said camera units into other coordinate system using perspective 
transformation; means of composi ting multiple transfo rmation images into a 
single image relative to the image of thej ghicle; and a display that shows said 
image to an occupant of the vehicle. 
(Function) 

A camera image for example is transformed into an image on a plane coordinate 
system, on the coordinate origin of which the image of the vehicle is displayed. 
Since viewing this display allows the vehicle occupant to be aware of relationship 
between the vehicle's travelin g direction and the situational surroundings, the_ 
occupant can take appropriate measures. 
(Example) 

Hereinafter, one embodiment of this invention is explained based on drawings 
shown in Fig. 1 - Fig. 4. 

Fig. 1 is a plain view of the vehicle equipped with camera units of this invention. 
Firstly it is constituted in such that the multiple camera units 1 - 6 having the 
projection ranges (fields of view) of 1a- 6a respectively are installed in the 
manner with which to cover the vehicle-surrounding environment as much as 
possible. That is, 2 units each at front and rear bumpers, 1 unit each at front 
winkers are mounted. Alternatively, rear combi lamps may be installed instead of 
camera units at a rear bumper. Inputting a reflected image into the camera unit 
by placing a mirr or in front of the camera lens allows for easier installation of 
camera units, which in turn increases the level of freedom in shaping the images. 
In addition, the number of camera units to be installed may be one or more, and 



moreover camera units should be installed slightly downward placing more 
importance on the road surface. 

Fig. 2 is a block diagram showing the constitution of the cameras projection 
image processing device in Fig. 1, in which images captured by the camera units 

I - N are entered into the image transformation part 7, in which the images are 
transformed into other coordinate system by perspective transformation and 
further composited into a single image at the image display part 8. The resulting 
image is displayed on the TV monitor 9, which is mounted at the driver seat. It is 
preferable that the image transformation part 7 has a built-in high-speed 
processor capable of parallel processing images captured by "N" camera units. 
At the image display part 8, the vehicle position is plotted by a simultaneous 
illustration, however the position at which the vehicle is displayed may be 
adjusted and moved from the coordinate origin according to the signals based on 
the gear position, the vehicle speed and the winker operation, so that the wider 
vehicle surrounding environment area can be viewed. Fig. 3 shows an_ example 
of the display results on these screens, ali of which are displayed on the plane 
(road surface) coordinate system. Fig. 3a shows a vehicl e in rever s ejjear (in th e 
arrow direction), in which the vehicle 10 is statically displayed at the upper center 
and the other vehicle 1 1 adjacent to the vehicle is displayed in the side view. Fig. 
3b shows a vehicle during the right winker operation, in which the other vehicle 

I I at the rear side and in the same direction of the vehicle can be displayed 
assuming the lane change (in the arrow direction) to the right. 

Moreover, Fig, 3c shows an example of a vehicle stopped at the junction where 
the view was obstructed by the wall 12 and due to the other vehicle 11, the 
vehicle is about to stop wherein another vehicle 1 V at the rear side of the vehicle 
is also displayed. 

Fig. 4 is a processing flow chart of the image transformation part 7 in Fig. 2, by 
which the processing is explained. 

The images captured by the camera units 1 - N are simultaneously entered into 
the image transformation part 7 and the resulting image is defined as A1 (X, Y). 
"1" denotes the camera unit number. Then the perspective transformation 
(coordinate transformation) of "N" images is carried out in parallel, and the 
resulting image is defined as B1 (X, Y), based on which the following equations: 
B1 (X, Y) = A1 (X,Y) 



a, X + b < Y * c r 

X d , X + e . Y + f i 

g , X + h , V + fc . 

Y d , X + e , Y + t , 

can be obtained. In this formula, A1, b1, d, d1, e1, f1, g1, hi, and k1 denote the 
focal length of camera unit, which is a parameter defined by the installation 
position of each camera unit (an angle against the vehicle, an angle against the 
road surface etc.), which is uniquely defined upon the installation and can be 
memorized. The perspective transformation converts the screen coordinate 
system of camera unit into the road surface (plane) coordinate system wherein 
the vehicle center is at the coordinate origin, left and right sides of the vehicle 
traveling direction is plotted on X-axis, and the vehicle traveling direction is 
plotted on Y-axis. 

Then, at the image display part 8, each image transformed into the above 
mentioned coordinate system is constituted. 

In order to do so, firstly the display position of the vehicle is determined 
according to the winker operation, the gear position and the vehicle speed, 
defining the coordinate as (XO t YO). Then the monitor image C (X,Y) is cleared 
and the vehicle is illustrated around the center point (XO.YO). Then the camera 
transformation image B1 (X,Y) (I = 1 - N: a number of camera units) is transferred 
toC(X,Y). 

At this conjuncture, as for the overlapping fields of view, the average value of the 

density can be assigned. That is, following equations are applicable. 

C (X,Y) = B1 (X - X0,Y - YO) . when C (X,Y) = 0. 

C (X,Y) = C (X,Y) . when B1 (X - X0,Y - YO) = 0. 

C (X,Y) = 1/2 (C (X,Y) + B1 (X - X0,Y - YO)) .in other cases. 

Hereto, 

I = 1 - N 

X = 1 - Mx (number of pixels of display images in the lateral direction ) 
Y = 1 - My (number of pixels of display images in the vertical direction ) 
Based on the above, both of transformed images from camera units and the 
illustrated image of the vehicle can be displayed together on the image C (X,Y). 
Thus, viewing the above display enables the driver (and other occupants) to 
appropriately prepare for: backing a vehicle in Fig. 3a; changing a lane to the 



right-turn lane in Fig 2b; or passing through an impenetrable junction. In addition, 
the transformation coordinate is not limited on the road surface only, but for 
example, a plane at a certain height on the road surface may also be included. 
Furthermore, since the position at which the vehicle is displayed may be adjusted 
and moved from the center (coordinate origin) according to the conditions such 
as the gear position and the winker operation, the wider vehicle surrounding 
environment area can be displayed. Moreover, during the high speed driving, the 
TV monitor image can be reduced to display wider area, by which more effective 
images can be obtained. 

Moreover, a color TV may be used to display images in color, whereby objects 
are more recognizable. 

Moreover, at night, an infrared camera may be used to easily identify obstacles 
under insufficient illumination. 
(The Effect of the Invention) 

As described herein above, since the constitution according to this invention 
comprises of one or multiple camera units mounted on a vehicle; means of 
transforming images captured by said camera units into other coordinate system 
using perspective transformation; means of compositing multiple transformation 
images into a single image relative to the image of the vehicle; and a display that 
shows said image to an occupant of the vehicle, the correlation between the 
vehicle and the situational surroundings can be better recognized, thereby the 
occupant can take appropriate measures effectively. 
4. BRIEF EXPLANATION OF DRAWINGS 

Fig. 1 is a plain view of a vehicle showing an example of the positioning of 
camera units according to this invention; Fig. 2 is the configuration block diagram 
of the apparatus according to this invention; Fig. 3 is^anjx ample of TV monitor 
display result according to this invention wherein (a) shows a vehicle in reverse 
gear, (b) shows a vehicle during the right winker operation, and (c) shows the 
image of a vehicle stopped; Fig. 4 is a image processing flow chart of a block 
diagram in Fig. 2; Fig. 5 shows an conventional example wherein (a) is an 
arrangement plan of TV camera units; (b) is a processing configuration drawing; 
and (c) is an example of images. 
1, 2, 3, 4, 5, and 6: Camera unit 
7: Image transformation part 



8: Image display part 
9: TV monitor 
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